Flow stability is one of the classical problems in fluid mechanics and atmosphere–ocean dynamics,^[1–4] and the linear stability theory is the most mature. Two main approaches are usually adopted in the field of flow stability. The first one is the normal method. The most famous application is the linear instability criterion of incompressible flow, i.e., Rayleigh’s criterion (it is the necessary conditions of instability, since there exist inflection points in the basic flow , it can be called inflection point theorem). Kuo^[5] applied this method to meteorology and derived atmospheric barotropic stability criteria in 1949. Pedlosky^[6] studied the atmospheric baroclinic stability. The second one is the variational method along with eigenvalue estimate, which was proposed by Synge et al.^[1–3] when studying the Orr–Sommerfeld equations’ eigenvalue problem of viscous fluid. Chaney and Eady studied the baroclinic instability problems in the process of atmospheric movement.^[7] Oliver and Ulrich^[8] discussed the linear stability of compressible flow, and solved the eigenvalue problem of linear stability theory for three different cases. Wang and Huang^[9] used the variational method to deduce the sufficient conditions of linear stability and instability when studying ionospheric variations in equatorial and low latitude areas.

With the development of linear stability, the theories of weak nonlinear and nonlinear stability were also developed accordingly. Among them, the perturbation method is widely used, which can be referred to Refs. [6] and [7].

In the development of nonlinear stability theory, Arnol’d^[10,11] is the first one to deduce the nonlinear stability of planar inviscid steady incompressible flow through the variational method along with priori estimate, which opens a new phase for the research. Afterwards, based on Arnol’d’s idea, Mu et al.^[12,13] did many valuable researches when studying multilayer quasi-geostrophic flow and continuous stratified quasi-geostrophic flow. They took the generalized energy conservation and generalized energy–enstropy conservation into consideration, established a new conserved term, and derived Arnol’d’s first and second theorems by making use of the prior estimate and Poincare inequality. Zhang and Xiang^[14,15] investigated nonlinear saturation of baroclinic instability in the generalized Phillips model in the context of Arnol’d’s second theorem and obtained the upper and lower bounds on the disturbance energy and potential enstrophy in the case of nonlinearly unstable basic flow. Based on the previous work, Swaters^[16] studied linear and nonlinear stability of the barotropic vorticity equation by using a Hamiltonian system, and gave a new idea for stability research.

As we all know, the Hamiltonian principle can be counted as the greatest principle not only in mechanics, but even in the whole of physics. It integrally describes the motion of a mechanical system and reveals basic laws that energy transformation should follow. At the same time, because of its uniformity and conciseness, the Hamiltonian principle has excellent universality. For the above reasons, many scholars are devoted to studying it.^[17–21] However, it is a difficult work to transform nonlinear partial differential equations into an infinite-dimensional Hamiltonian system, since the following two parts must be accomplished. The first one is the establishment of a conservative term H and a differential operator J, which is the key to transform nonlinear partial differential equations into the form ; the second one is the definition of Poisson bracket , such that J satisfies the five properties of Poisson bracket (self-Poisson commutation, skew-symmetry, distributive property, associative property, and Jacobi identity).

In this paper, the linear stability of multilayer quasi-geostrophic flow is discussed. First, the multilayer quasi-geostrophic flow nonlinear equations are turned into the form of an infinite-dimensional Hamiltonian system , the problem of finding the operator J is involved, which is related to the Poisson bracket. And then, Hamiltonian function H₂ is required to be conservative, and its gradient should be calculated. After these works, a Hamiltonian system can be established. Second, basic flow is introduced. Since J is a differential operator and irreversible, the equation can not be derived from the expression , namely, cannot be an extreme point of Hamiltonian function H₂. It is essential to reconstitute Hamiltonian function H, which should satisfy the following three conditions: (i) H is time independent, i.e. , (ii) H concludes an extreme point , (iii) H maintains the Hamiltonian system invariant. From the analyses above, H contains not only the kinetic energy of the system, the potential vorticity energy, the boundary circulation energy, but also the basic flow energy. Third, general disturbances are superposed to the basic flows, and the linearized equations of multilayer quasi-geostrophic flow are derived. Unlike nonlinear stability, now the linearized equations do not have a Hamiltonian structure, and we do not make use of the normal method to discuss the stability. Fourth, stability criteria are established. The deduction of the second order variation of is the key, that is , and it is necessary to prove that it is invariable over time, i.e., . Based on these works, the linear stability criteria in the sense of Liapunov are derived for two different cases that the basic flow is a maximum point ( and a minimum point ( of the new Hamiltonian function H by using the methods of prior estimate and eigenvalue estimate.

For the nonlinear stability, Mu et al.^[12,13] have made a lot of contributions by taking advantage of the Arnol’d method, but so far no one has discussed multilayer linear stability on the basis of the Hamiltonian system. It should be noted that the results of nonlinear stability are not completely suitable for the linear stability. There exist many essential difficulties, so it is necessary to study linear stability.

Consider a stably stratified fluid of N superimposed layers of constant densities , with equal density jumps , and mean layer depth d_i. The multiplayer quasi-geostrophic vorticity equations are given by^[6] 1 2 where ψ_i and q_i are the stream function and the potential vorticity of the i^th layer, respectively, and F_i denotes the Froude number, i.e., . It is obvious that (a is a constant) and follows 3 where denotes the lowest layer orographic function, δ_iN is the Dirac function, f₀ and β are the Coriolis parameters, respectively, and is given by 4 By calculation, we have , so is a nonnegative symmetric matrix and .

In Eqs. (1) and (2), the region presents a multiply (or simple) connected domain in plane, is the boundary of , which is smooth, consists of m + 1 simple closed curves , and , and denotes the unit outward normal vector on the boundary curve .

The boundary conditions of the fluid are as follows. The boundary conditions of the rigid wall are 5 The conservation conditions of the boundary circulation are 6

If the following expressions in vector form 7 are introduced into Eqs. (1) and 2then we have 8a 8b 8c 8d

Lemma 1 For , suppose ψ, q, and φ are differentiable functions in , then we can obtain 9

Proof: We have 10 Specially, under the condition of ψ = 1, we obtain 11 Then the following expression 12 is established if the condition is introduced into Eq. (11).

In the process of establishing infinite-dimensional Hamiltonian system , there are three problems to be solved. The first one is the establishment of J that J satisfies the properties of the Poisson bracket; the second one is the conservative property of H, i.e., ; and the last one is the gradient calculation of H, that is , which should ensure that the equation is in accordance with the original nonlinear problem.

For the given basic flows and , they satisfy the second order variation of the Hamiltonian function , that is, is a minimal value of , and then the linear stability criterion can be deduced, i.e., Arnol’d’s first linear stability criterion.

Firstly, we derive the expression , and then prove that it is time independent, i.e., , the linear stability criterion under the condition that is the minimal value of with respect to the norm can be obtained.

Lemma 2 Suppose is a Hamiltonian function, then 47a 47b

Proof: Based on the expressions (32) and (29), we have 48 It follows from Eqs. (26), (39), (42) and that 49 50 Therefore, it can be concluded that 51

Lemma 3 is time independent, i.e., .

Proof: Based on the above result of Eq. (51), we obtain 52 It follows from Lemma 1 and Eq. (46) that 53 where the formulas and have been used.

The following result can be deduced from Lemma 1, , and : 54 From the formula , we have 55

Theorem 1 Suppose and satisfy 56 57 58 Denote 59 Then, it can be concluded that the basic flow is stable in the sense of Liapunov with respect to the norm .

Expression (59) presents the sum of perturbation energy and energy–enstropy.

Proof: Direct calculation of gives 60 Based on the assumption that is a positive definite symmetric matrix, and is a nonnegative matrix, we obtain 61 On the other hand, it can be deduced that from Lemma 3. Denote 62 It follows that 63 Thus, for every and for all , when , we have , i.e., the linear stability of holds in the sense of Liapunov with respect to the energy–enstropy norm (59).

For the given basic flows and , they satisfy the second order variation of the Hamiltonian function , that is, is a maximal value of , and then the linear stability criterion can be deduced, i.e., Arnol’d’s second linear stability criterion. Actually, the linear stability criterion is more meaningful than the first one in physics.

Lemma 4 Suppose is as indicated earlier, φ satisfies 64 then there exist denumerable eigenvalues , φ_n are the corresponding eigenfunctions, consist of the complete orthogonal system of , where c₀ is a constant and φ_n satisfies . So for an arbitrary , it can be expanded as the following form: 65 If , then we have , the proof can be referenced in the theory of partial differential equations.

Since is a nonnegative symmetrical matrix, denote the eigenvalues of as α_i, and they satisfy , thus, there is an orthogonal matrix , such that . From the theory of algebra, and have the same eigenvalues. Denote 66

Lemma 5 Suppose , and , satisfies 67 then the following inequality 68 can be derived.

Proof: Step 1 When N = 1, take the following boundary problem into account: 69 Based on and Lemma 4, can be expanded as follows: 70 then we obtain 71 So it follows that 72

Step 2 Substituting the expressions and into the first equation of Eq. (67) yields the following formula: 73 From , we can deduce , and based on step 1, it can be concluded that 74

Theorem 2 Suppose the basic flows and satisfy 75 76 77 where , ; 78 79 Denote 80 Then the basic flow is stable in the sense of Liapunov with respect to the norm .

The formula (80) presents the perturbed enstropy.

Proof: Step 1 If we can prove , then is a maximal value of . Calculate 81 Based on Lemma 5 and the expression of 82 we can obtain 83 From Eq. (78), it can be obtained that , therefore , namely, is a maximal point of .

Step 2 Estimate . Denote as the minimal eigenvalue of , and as the maximal eigenvalue of , then 84 85 So 86 Thus, for every and for all , when , we have , i.e., the linear stability of holds in the sense of Liapunov with respect to the enstropy norm (80).

Remark 1 In Theorem 2, we assume that the disturbance vorticity satisfies the condition (79) , while in the original problem. Actually, the condition (79) is not essential for the reason that we can divide the disturbances into , such that satisfies Eq. (79) and satisfies where denotes the original stream function value, , and presents the measure of . Now, the problem can be divided into two parts as follows: and Estimate and respectively, and the stability criterion can be deduced similarly.

Remark 2 When discussing the nonlinear stability, the functional is introduced to substitute . First of all, we should calculate and the conserved quantity of , i.e., , then on the basis of the estimation of the operator , the stability criteria can be obtained, too.

The linear stability of multilayer quasi-geostrophic flow is studied in this paper. Firstly, the equations are turned into the form of an infinite-dimensional Hamiltonian system . To make the basic flow an extreme point of the Hamiltonian function, a new Hamiltonian function H is reconstructed. It is time independent and maintains the structure of the system invariant, namely, , . Secondly, the linearized equations are derived by superposing infinitesimal disturbances on the basic flows. Finally, the linear stability criteria are deduced in the sense of Liapunov under two different conditions with respect to certain norms. That is, the basic flow is a minimum point or a maximal point of H, the energy–enstropy and the enstropy are introduced to be norms, respectively. The nonlinear stability can be researched by using the same method, which we will discuss in other articles.